Are Latent Sentence Vectors Cross-Linguistically Invariant?

Previous work [Bowman et al., 2016] has shown that variational autoencoders (VAEs) can create distributed representations of natural language that capture different linguistic levels such as syntax, semantics, and style in a holistic manner. I investigate to what extent VAEs, when trained on different languages, result in comparable representations. To this end, I train VAEs for English and French, and then train a transformation between the resulting latent spaces on the task of machine translation. An analysis of the resulting mapping from French to English sentences shows that the latent representations represent the presence of words, phrases, and the general topic. However, I do not find evidence that they also encode syntax and semantics in a cross-linguistically invariant manner.